Advances in compression techniques for audio-visual information have resulted in cost effective and widespread recording, storage, and/or transfer of movies, video, and/or music content over a wide range of media. The Moving Picture Experts Group (MPEG) family of standards is among the most commonly used digital compressed formats. A major advantage of MPEG compared to other video and audio coding formats is that MPEG-generated files tend to be much smaller for the same quality. This is because MPEG uses very sophisticated compression techniques. However, MPEG compression may be lossy and, in some instances, it may distort the video content. In this regard, the more the video is compressed, that is, the higher the compression ratio, the less the reconstructed video resembles the original information. Some examples of MPEG video distortion are a loss of texture, detail, and/or edges. MPEG compression may also result in ringing on sharper edges and/or discontinuities on block edges. Because MPEG compression techniques are based on defining blocks of video image samples for processing, MPEG compression may also result in visible “macroblocking” that may result due to bit errors. In MPEG, a macroblock is the area covered by a 16×16 array of luma samples in a video image. Luma may refer to a component of the video image that represents brightness. Moreover, noise due to quantization operations, as well as aliasing and/or temporal effects may all result from the use of MPEG compression operations.
When MPEG video compression results in loss of detail in the video image it is said to “blur” the video image. In this regard, operations that are utilized to reduce compression-based blur are generally called image enhancement operations. When MPEG video compression results in added distortion on the video image it is said to produce “artifacts” on the video image. For example, the term “mosquito noise” may refer to MPEG artifacts that may be caused by the quantization of high spatial frequency components in the image. Mosquito noise may also be referred to as “ringing” or “Gibb's effect.” In another example, the term “block noise” may refer to MPEG artifacts that may be caused by the quantization of low spatial frequency information in the image. Block noise may appear as edges on 8×8 blocks and may give the appearance of a mosaic or tiling pattern on the video image.
Some of the characteristics of mosquito noise may result from the fact that it is an artifact of the 8×8 block Discrete Cosine Transform (DCT) operation in MPEG compression. While generally confined to a particular 8×8 block of video samples, in some instances, motion compensation may result in mosquito noise beyond the block boundary. Mosquito noise commonly appears near luma edges, making credits, text, and/or cartoons particularly susceptible to this form of artifact. Mosquito noise may be more common, and generally more severe, at low bit rates. For example, mosquito noise may be more severe when macroblocks are coded with a higher quantization scale and/or on a larger quantization matrix.
Mosquito noise may tend to appear as very high spatial frequencies within the processing block. In some instances, when the input video to the MPEG compression operation has any motion, the mosquito noise generated may tend to vary rapidly and/or randomly resulting in flickering noise. Flickering noise may be particularly objectionable to a viewer of the decompressed video image. In other instances, when the input video to the MPEG compression operation is constant, the mosquito noise that results is generally constant as well. Horizontal edges tend to generate horizontal ringing while vertical edges tend to generate vertical ringing. While mosquito noise may also occur in the color components or chroma of a video image, it may generally be less of a problem since it is less objectionable to a viewer of the decompressed video image.
Some of the characteristics of block noise may result from the fact that it is an artifact of the 8×8 block discrete cosine transform (DCT) operation in MPEG compression. Block noise may generally occur near a block boundary. While block noise may occur anywhere on an image, it is more commonly seen in nearly smooth regions, such as the sky and faces, or in high motion or high variance regions, such as moving water. Block noise may be more common, and generally more severe, at low bit rates. For example, block noise may be more severe when macroblocks are coded with a higher quantization scale and/or on a larger quantization matrix. While block noise is typically caused by quantization of low spatial frequency terms that result from the DCT operation, it is not generally caused by the quantization of the DC term. For example, MPEG compression generally provides at least 8 bits when quantizing the DC term of intra coded blocks.
Block noise may also appear at discontinuities located at or near the block edges. The block boundaries may remain fixed even when the video image moves. In this regard, a static block pattern may stand out strongly against a moving background, a condition that may be highly objectionable from a viewer's perspective. In some instances, however, motion vectors generated during MPEG compression may cause block noise to move with the video image, but this is generally less common and less objectionable from a viewer's perspective. Block noise may be more objectionable on vertical edges than on horizontal edges, particularly on an interlaced display.
Block noise may generally be more pronounced in certain picture coding types. For example, block noise may be often worse in intra coded pictures or I-pictures and in predicted pictures or P-pictures. While block noise is generally associated with the luma component of a video image, it may also occur in the chroma component of a video image. However, the block noise in the chroma component may generally be less of a problem since it is less objectionable to a viewer of the decompressed video image.
There have been attempts to provide normative approaches for reducing the effects of mosquito noise. For example, the MPEG4 specification ISO/IEC 14496-2:1999/Amd.1:2000(E) Annex F comprises a state-of-the-art mosquito noise filter, which is also called a deringing filter, which may be utilized to filter out mosquito noise. However, the MPEG4-based deringing filter may have several limitations. For example, the MPEG4 deringing filter may have a hard threshold based on the binary index operation bin(h,v). Accordingly, small changes in pixel values may cause the filter to turn ON or OFF, causing objectionable pixel flickering. The MPEG4 deringing filter may only be applied to 8×8 blocks. This may limit the utility of the deringing filter since under high-motion and/or low bit rate conditions motion compensation may move mosquito noise beyond the transform block edges. The deringing filter kernel is symmetrical vertically and horizontally and as a result, the deringing filter may not correct for interlaced video, where the vertical pixel or sample distance is twice the horizontal pixel distance. Another limitation arises because the detection algorithm utilized by the MPEG4 deringing filter may often overfilter or underfilter video images. Moreover, the detection algorithm may utilize a 10×10 block of pixels or samples to detect mosquito noise and this large block size may be very expensive for raster-scan implementations.
The MPEG4 specification ISO/IEC 14496-2:1999/Amd.1:2000(E) Annex F also comprises a state-of-the-art block noise filter, which is also called a deblocking filter. However, the MPEG4-based deblocking filter may have several limitations. For example, the block noise detection algorithm utilized by the deblocking filter is based only on a few neighboring pixels and, as a result, may frequently filter real content erroneously. Horizontal edges may be filtered utilizing a similar approach as with vertical edges, making this method inappropriate for interlaced video and resulting in artifacts caused by the filtering process. Moreover, horizontal edge filtering may require 10 vertically adjacent pixels, for example. For raster-scanned video this may require 9 lines stores, which may make the implementation very expensive.
Future solutions to the presence of these types of video compression artifacts may need to provide cost effective and easy to implement reductions in mosquito and block noise without any perceptible degradation in video quality. Furthermore, even though block noise and mosquito noise may be reduced, the manner in which they are realized may result in perceptible degradation of video quality.
Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings.